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ARI  Research  Reports  and  Technical  Papers  are  intended  for  sponsors  of 
R&D  tasks  and  other  research  and  military  agencies.  Any  findings  ready  for 
implementation  at  the  time  of  publication  are  presented  in  the  latter  part  of 
the  Brief.  Upon  completion  of  a major  phase  of  the  task,  formal  recommen- 
dations for  official  action  normally  are  conveyed  to  appropriate  military 
agencies  by  briefing  or  Disposition  Form. 


FOREWORD 


The  Human  Factors  Technical  Area  is  concerned  with  the  demands  of 
increasingly  complex  battlefield  systems  that  are  used  to  acquire,  trans- 
mit, process,  disseminate,  and  utilize  information.  This  increased  com- 
plexity places  greater  demands  upon  the  operator  interacting  with  the 
machine  system.  Research  in  this  area  focuses  on  human  performance 
problems  related  to  interactions  within  command  and  control  centers  as 
well  as  issues  of  system  development.  It  is  concerned  with  such  areas 
as  software  development,  topographic  products  and  procedures,  tactical 
symbology,  user-oriented  systems,  information  management,  staff  opera- 
tions and  procedures,  decision  support,  and  sensor  systems  integration 
and  utilization. 

An  issue  of  special  concern  within  the  area  of  user-oriented  sys- 
tems has  been  the  improvement  of  manual  data  input  procedures,  especially 
in  the  Tactical  Operations  System  (TOS) . The  main  source  of  information 
for  tactical  data  systems  is  manual  data  entry — a slow,  error-prone  pro- 
cess. The  capability  of  tactical  data  systems  such  as  TOS  to  support 
command  staff  actions  with  accurate,  complete,  and  timely  information 
depends  on  the  performance  of  the  human  operator  who  must  manually  enter 
information  into  the  system.  Previous  ARI  research  on  data  entry  has 
resulted  in  simplified  message  formats,  improved  reference  codes,  and 
aids  for  on-line  preparation  and  verification  of  message  entries.  Al- 
though progress  has  been  made,  data  entry  remains  a major  system  bottle- 
neck. The  research  reported  here  demonstrated  the  effectiveness  of  a 
response-sensitive  training  strategy  in  reducing  the  amount  of  time  re- 
quired for  operator  training. 

Research  in  the  area  of  user-oriented  systems  is  conducted  as  an 
in-house  effort  augmented  through  contracts.  The  current  report  re- 
sulted from  an  in-house  research  effort  initiated  under  the  program  di- 
rection of  Mr.  James  D.  Baker  and  responsive  to  requirements  of  Army 
Project  2Q763743A774  and  to  special  requirements  of  the  U.S.  Army  Com- 
bined Army  Combat  Development  Activity,  Fort  Leavenworth,  Kans.  Special 
requirements  are  contained  in  Human  Resource  Needs  75-150,  "Online  Aids 
for  Tactical  Data  Inputting." 
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BRIEF 


Requirement : 

To  evaluate  alternative  on-line  computer-assisted  training  strate- 
gies for  improving  performance  of  the  Message  Input  Output  Device  (MIOD) 
operator  in  the  Tactical  Operations  System  (TOS) . 


Procedure : 

MIOD  training  was  given  to  53  Army  enlisted  personnel  under  one  of 
four  training  methods:  minimum  feedback,  edit  feedback,  remedial  feed- 
back, and  response-sensitive  feedback.  The  first  three  methods  were 
based  on  the  traditional  linear  instructional  strategy  model,  and  the 
fourth  method  was  based  on  a response-sensitive  model  of  instruction. 

The  transfer  of  this  training  was  tested  under  two  different  operational 
configurations  of  TOS:  no  feedback  and  edit  feedback. 


Findings : 

Use  of  a response-sensitive  instructional  strategy  markedly  re- 
duced training  time.  No  differences  in  data  entry  speed  or  accuracy 
were  found  among  the  four  training  conditions,  or  for  participants 
tested  in  each  of  the  two  operational  configurations.  Mean  entry  accu- 
racy was  more  than  80%.  Most  of  the  errors  (more  than  90%)  were  types 
that  could  not  be  detected  by  computer  edit  routines. 


Utilization  of  Findings: 

A response-sensitive  training  strategy  is  an  effective  method  for 
reducing  MIOD  training  time  without  sacrificing  inputting  accuracy. 

The  decision  to  adopt  a response-sensitive  training  strategy  for  spe- 
cific training  should  consider  the  trade-off  between  the  added  cost  of 
training  development  and  the  cost  savings  of  reduced  training  time. 

Development  of  more  elaborate  edit  routines  may  help  reduce  errors 
but  probably  will  not  eliminate  them.  Improved  formats  and  definition 
of  legal  entries  may  provide  an  effective,  simple  method  to  reduce  in- 
put errors. 
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SELECTIVE  FEEDBACK  AS  A TRAINING  AID  TO  ON-LINE 
TACTICAL  DATA  INPUTTING 


INTRODUCTION 

The  primary  function  of  the  Army's  evolving  semiautomated  Tactical 
Operations  System  (TOS)  is  to  provide  commanders  and  their  staffs  with 
current,  accurate  battlefield  information  on  which  to  base  tactical  de- 
cisions. The  effectiveness  of  TOS  in  providing  timely  and  accurate 
information  is  heavily  dependent  on  the  performance  of  the  Message 
Input  Output  Device  (MIOD)  operator  who  must  manually  enter  tactical 
information  into  the  system.  During  the  evolution  of  TOS,  it  became 
apparent  that  data  inputting  was  perhaps  the  most  crucial  component  of 
the  system.  Delays  or  errors  in  data  entry  :an  have  a devastating  im- 
pact on  the  quality,  completeness,  and  timeliness  of  the  subsequent 
processing  of  information  by  the  system.  Data  entries  also  need  to  be 
relatively  error-free,  since  more  than  80%  of  operator  errors  probably 
will  enter  the  system  undetected  (Strub,  1975) . 

Previous  research  (reviewed  by  Alderman,  1975)  focused  on  im- 
proving the  data  inputting  performance  (reducing  time  and  errors)  of 
MIOD  operators  by  reallocating  the  inputting  functions/tasks  and  devel- 
opment of  job/training  aids.  However,  if  data  entry  time  as  well  as 
the  number  of  data  entry  errors  are  to  be  reduced  further,  MIOD  opera- 
tors must  be  trained  to  become  more  proficient. 

The  problem  of  training  a human  operator  for  effective  on-line 
data  input  to  the  TOS  is  threefold:  initial  training  of  the  MIOD  op- 
erator, optimum  transfer  of  new  skills  to  operational  settings,  and 
maintenance  of  the  high  levels  of  proficiency  developed  during  the 
first  two  phases.  One  approach  to  training  MIOD  operators  is  to  devel- 
op training  modules  that  can  be  embedded  within  the  TOS  software. 
Modules  could  be  used  to  train  operators  and  to  maintain  trained  op- 
erators at  a high  level  of  proficiency.  The  effectiveness  of  an  em- 
bedded training  module  is  highly  dependent  upon  development  of  opti- 
mizing training  strategies.  Such  optimizing  strategies  should  be 
arranged  to  accommodate  individual  differences  among  learners  (e.g., 
educational  history,  current  skill  level)  as  well  as  quantitative  and 
qualitative  differences  in  the  material  to  be  learned  (Atkinson  & Paul- 
son, 1972). 

A response-sensitive  instructional  strategy  provides  the  basis 
for  an  instructional  system  that  accommodates  differences  in  individu- 
als as  well  as  differences  in  the  material  to  be  learned  (Atkinson, 
1972) . Briefly,  this,  strategy  uses  a trial-by-trial  response  history 
to  assess  the  student's  current  state  of  learning  so  that  further  in- 
structional items  can  be  selected  to  optimize  learning.  To  illustrate 
how  this  strategy  works,  assume  you  wish  to  train  potential  MIOD 
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operators  to  use  the  specific  message  input  format  EC1  (see  the  Ap- 
pendix for  format  listings).  During  the  initial  stages  of  training, 
you  would  begin  to  select  items  for  further  training  based  on  the  op- 
erator’s performance  history.  For  example,  suppose  after  giving  three 
EC1  format  messages  you  observed  that  a particular  student  never  made 
a mistake  on  the  sections  labeled  AGENCY  and  SOURCE,  but  performed  in- 
consistently on  other  sections  of  the  format.  Under  a traditional  or 
linear  strategy  you  would  continue  to  give  this  student  practice  on 
all  items.  Under  a response-sensitive  strategy  you  would  reduce  the 
frequency  of  student  practice  (possibly  to  zero)  on  the  sections  labeled 
AGENCY  and  SOURCE  and  concentrate  the  training  on  those  items  where 
performance  has  been  inconsistent. 

Strub  (1971)  has  shown  that  inputting  accuracy  can  be  increased 
when  MIOD  operators  use  a cathode-ray  tube  (CRT)  to  input  data  directly 
into  the  TOS  computer  system.  He  also  found  that  entry  times  could  be 
decreased  by  providing  a computer-generated  format  on  the  CRT  as  an 
aid  to  data  entry  (Strub,  1975).  Strub  (1975)  suggested  that  an  auto- 
mated training  program  incorporating  these  inputting  aids  with  a 
response-sensitive  instructional  strategy  might  be  used  to  train  MIOD 
operators  more  effectively.  The  present  experiment  was  designed  to 
investigate  the  relative  effectiveness  of  following  Strub' s suggestion 
of  using  these  inputting  aids  within  the  framework  of  a response- 
sensitive  instructional  strategy  to  train  MIOD  operators. 

It  was  hypothesized  that  use  of  a response-sensitive  instructional 
strategy  in  conjunction  with  computer-generated  feedback  would  have  two 
desirable  effects  on  MIOD  operator  training.  First,  it  was  expected 
that  such  a strategy  would  allow  subjects  to  progress  through  the  train- 
ing material  at  a faster  rate  than  would  other  more  conventional  strate- 
gies. Second,  it  was  thought  that  a response-sensitive  strategy  would 
produce  better  transfer  of  training  to  operational  environments  than 
would  other  training  strategies.  This  enhancement  of  transfer  of  train- 
ing was  expected  to  result  in  greater  input  accuracy  scores  and/or  faster 
entry  times  during  transfer-testing. 

The  objective  of  the  research,  then,  was  to  assess  the  impact  of 
response-sensitive  training  on  the  speed  and  accuracy  of  tactical  data 
inputting  by  MIOD  operators  in  a TOS  environment. 
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METHOD 


Participants 

The  participants  were  71  enlisted  personnel  who  volunteered  to 
serve  for  7 to  8 hours  a day  for  2 consecutive  days.  To  approximate 
the  capabilities  necessary  for  assignment  as  an  MIOD  operator,  par- 
ticipants were  requested  who  had  General  Technical  (GT)  scores  of  105 
or  higher  and  who  had  normal  vision  or  vision  correctable  to  20/20. * 


Apparatus 


The  experiment  was  conducted  on-line  in  four  separate  work  sta- 
tions. Each  station  contained  an  IBM  1050  printer/keyboard  terminal 
and  a CDC  210  CRT  display  and  associated  keyboard  entry  panel. ^ The 
1050  terminals  printed  out  free-text  messages  that  simulated  battle- 
field information.  Messages  consisted  of  two  general  types,  those 
concerned  with  enemy  activities  and  those  concerned  with  the  activities 
of  friendly  forces.  The  CRT's  displayed  the  formats  into  which  par- 
ticipants entered  information  translated  from  the  free-text  messages. 
All  information  about  enemy  activities  used  the  same  format.  Friendly 
activities,  however,  required  that  the  participant  choose  one  of  nine 
other  formats  based  on  the  subject  matter  of  the  free-text  message. 

With  slight  modifications,  the  messages  and  the  formats  used  in  the 
present  experiment  were  the  same  as  those  used  by  Strub  (1975).  Copies 
of  the  10  formats  (9  friendly,  1 enemy)  appear  in  the  Appendix  along 
with  a list  of  the  order  in  which  they  appeared  in  the  experiment. 
Participants  were  given  a notebook  containing  a detailed  set  of  in- 
structions that  outlined,  step  by  step,  each  task  they  were  to  learn. 

In  addition,  participants  were  given  another  notebook  that  contained 
an  entry-by-entry  description  of  each  of  the  10  message  formats  and  a 
list  of  legal  entries  for  each  of  the  format  entry  items. 
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Procedure 

Each  participant  took  part  in  a training  session  and  a transfer- 
testing session,  which  were  conducted  on  2 consecutive  days.  After 
receiving  a brief  description  of  the  experiment,  participants  were 
randomly  assigned  to  one  of  the  four  work  stations  and  given  notebooks 
that  contained  detailed  descriptions  of  their  tasks,  legal  entry  tables, 
and  a scenario  that  described  the  simulated  combat  environment  in  which 
they  would  be  operating. 


No  attempt  was  made  to  verify  GT  scores  with  participant  records. 

, 

Commercial  names  are  used  for  purposes  of  clarity  only  and  do  not  in- 


Participants  were  given  a brief  keyboard  orientation  that  included 
a practice  drill  using  the  various  keyboard  functions.  This  was  fol- 
lowed by  a 12-item  typing  exercise  to  familiarize  them  with  the  type- 
writer output  and  CRT  keyboard  inputting  procedures.  After  the  partici- 
pants had  completed  both  the  keyboard  drill  and  typing  exercise  and  had 
read  the  detailed  instructions  for  the  experiment,  the  experimenter 
demonstrated  the  method  to  be  used  for  inputting  free-text  messages 
into  the  computer.  The  demonstration  consisted  of  a step-by-step  mes- 
sage analysis  and  a detailed  entry  procedure  for  a sample  friendly  mes- 
sage. The  procedure  was  demonstrated  to  each  group  of  four  participants 
on  a large  piece  of  posterboard  that  had  the  appropriate  format  inscribed 
on  it.  Participants  returned  to  their  work  stations  to  enter  the  sample 
message.  If  participants  failed  to  identify  the  correct  entry,  it  was 
supplied  by  the  experimenter  with  an  explanation.  Entries  might  be  cor- 
rect, incorrect,  or  illegal  (not  a member  of  the  appropriate  legal  entry 
table)  . 

Participants  were  required  to  complete  21  messages  during  training: 

15  training  messages  on  the  first  day  and  6 on  the  second  day.  Each 
participant  received  one  of  the  following  four  types  of  training  feedback: 

1.  Minimum  Feedback.  Participants  received  messages  on  the  lower 
part  of  the  CRT  whenever  illegal  or  incorrect  information  was 
entered.  The  error  message  informed  them  that  their  last  entry 
was  in  error  and  that  they  should  continue  to  the  next  entry. 
Participants  could  not  go  back  to  correct  their  mistakes. 

2.  Edit  Feedback.  An  error  message  was  provided  when  an  illegal 
entry  was  made.  The  participant  was  instructed  to  refer  to 
the  appropriate  legal  entry  tables  and  to  reenter  the  infor- 
mation before  proceeding  to  the  next  entry  item.  Incorrect 
entries  in  legal  format  produced  no  feedback. 

3.  Remedial  Feedback.  If  an  entry  was  incorrect,  an  error  mes- 
sage containing  both  the  incorrect  and  the  correct  entry  was 
displayed  at  th  bottom  of  the  CRT.  A correct  entry  was  re- 
quired before  the  participant  could  proceed  to  the  next  entry. 

4.  Response-Sensitive  Feedback.  This  condition  was  identical  to 
the  Remedial  Feedback  condition  with  the  following  exception: 
After  the  participant  had  entered  information  correctly  into 
a particular  element  of  a particular  type  of  format  three 
consecutive  times,  the  computer  would  fill  in  the  correct 
answer  for  the  participant  thereafter  when  that  element  of 
the  format  was  encountered  (auto-fill).  The  figure  of  three 
consecutive  correct  entries  was  chosen  as  the  criterion  for 
receiving  auto-fill  based  on  the  performance  of  20  pilot  sub- 
jects. This  figure  represents  the  median  number  (rounded  to 
the  highest  whole  number)  of  consecutive  errors  made  by  the 
pilot  subjects  on  all  common  entry  items  occurring  on  the  first 
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21  messages.  Because  none  of  the  friendly  formats  occurred 
more  than  three  times  during  the  training  session  (Appendix) , 
auto-fill  was  possible  only  for  enemy  messages. 

After  completing  the  training,  participants  were  briefed  about  the 
type  of  feedback  they  would  receive  during  transfer-testing.  Each  par- 
ticipant received  one  of  two  types  of  feedback  during  transfer-testing: 

1.  No  Feedback.  Under  this  condition  the  participant  was  never 
given  information  about  the  correctness  or  legality  of  his 
entries.  All  entries  were  accepted  by  the  computer. 

2.  Edit  Feedback.  This  condition  was  exactly  the  same  as  that 
for  Edit  Feedback  training. 

Participants  then  completed  as  many  messages  as  possible,  up  to  the 
maximum  of  43,  in  the  testing  session  which  lasted  the  rest  of  Day  2. 

A summary  of  the  error  feedback  features  for  each  of  the  feedback 
conditions  is  given  in  Table  1. 


Table  1 

Summary  of  Error  Feedback  Features 


Feedback 

condition 

Error 

report 

Correction 

required 

Correct 
entry  given 

Auto- 

fill 

No  feedback 

- 

- 

- 

- 

Minimum 

+ 

- 

- 

- 

Edit 

+a 

+ 

- 

- 

Remedial 

+ 

+ 

+ 

- 

Response- 

sensitive 

+ 

+ 

+ 

+ 

a 

Errors  in  Edit  Feedback  are  detected  by  computer  edit  and  validate 
routines  only. 
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Experimental  Design 

The  experimental  design,  shown  in  Table  2,  was  a 2 x 2 factorial 
with  two  appended  control  groups,  yielding  a total  of  six  independent 
gioups.  Each  of  the  71  participants  was  randomly  assigned  to  one  of 
these  groups. 

Table  2 

Experimental  Design 


Training  Mode 

Transfer-Testing  Mode 

No  Feedback  Edit  Feedback 

Experimental  condition 

Remedial  Feedback 
Response-Sensitive  Feedback 

na=  10  n = 9 

n = 8 n = 10 

Control  condition 

Minimum  Feedback 

Edit  Feedback 

n = 7 

n = 9 

aNumber  of  participants  in  each  group  who  completed  both  phases  of  the 
experiment  (N  = 53) . 


Independent  Variables.  The  two  between-group  independent  varia- 
bles were  Training  Mode  and  Testing  Mode.  The  Training  Mode  had  four 
conditions:  two  experimental  (Remedial  Feedback  and  Response-Sensitive 

Feedback)  and  two  control  (Minimum  Feedback  and  Edit  Feedback) . Data 
from  the  control  conditions  were  combined  for  analysis  and  the  result- 
ing combination  referred  to  as  the  Combined  Control  Feedback  condition. 
The  Testing  Mode  had  two  conditions:  No  Feedback  and  Edit  Feedback. 

In  addition  to  the  two  between-group  variables,  there  were  two 
within-group  independent  variables:  Message  Type  (friendly  and  enemy) 
and  Sessions  (training  and  testing)  . 

Dependent  Variables.  The  three  dependent  variables  were  mean 
time  per  entry,  mean  accuracy  of  first  entries,  and  mean  accuracy  of 
final  data  entries. 
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Mean  time  per  entry,  as  a function  of  training  and  testing  feed- 
back conditions,  was  the  average  time  to  complete  each  entry  item  of 
a single  message. 

Accuracy  of  first  data  entries  during  both  training  and  testing 
was  the  accuracy  of  the  first  response  a participant  made  for  each 
entry.  Accuracy  was  measured  as  a percentage: 

Number  of  correct  items 
Number  of  items  possible 


Accuracy  of  final  data  entries  was  measured  during  testing  only. 

If  a participant  made  an  error  that  was  caught  by  the  computer  (e.g., 
a misspelling)  and  that  the  participant  then  corrected  to  a computer- 
acceptable  form,  only  the  final  response  was  counted  as  correct  or  in- 
correct. Thus,  this  analysis  explored  the  quantity  of  errors  that  were 
not  detected  by  computer  validation.  If  a response  was  correct  the 
first  time,  of  course,  the  first  was  also  the  final  entry. 


RESULTS  AND  DISCUSSION 

To  assess  the  relative  effects  of  the  response-sensitive  instruc- 
tional strategy  on  speed  and  accuracy  of  performance  during  training 
and  transfer  of  training,  the  following  dependent  measures  were  analyzed: 
mean  time  per  entry,  mean  accuracy  of  first  entries,  and  mean  accuracy 
of  final  data  entries.  Mean  time  per  entry  and  mean  accuracy  of  first 
entries  were  analyzed  for  both  training  and  transfer-testing  phases  of 
the  experiment.  Mean  accuracy  of  final  data  entries  was  analyzed  only 
for  the  transfer-testing  phase.  Since  the  mean  time  per  entry  during 
the  12-item  typing  exercise  (given  at  the  beginning  of  the  experiment) 
showed  a significant  within-cell  correlation  with  each  of  the  dependent 
measures,  it  was  used  as  a covariate  in  all  analyses.3 


All  covariate  analyses  and  all  subsequent  simple  effects  tests 
were  accomplished  using  a significance  level  of  p = .05.  All  simple 
effects  testing  was  carried  out  using  either  analysis  of  variance  or 
Tukey's  HSD  test  where  appropriate  (Kirk,  1968). 


Typing-exercise  entry  time  was  positively  correlated  with  mean  time 
per  entry  (r  = +.57)  and  negatively  correlated  with  mean  accuracy  of 
first  data  entries  (r  = -.43)  and  mean  accuracy  of  final  data  entries 
(r  = -.44).  Typing-exercise  accuracy  did  not  show  any  significant 
within  cell  correlation  with  any  of  the  three  dependent  measures 
(r  * -.09,  r = +.10,  and  r * +.05  for  entry  time,  first  accuracy,  and 
final  accuracy,  respectively) . 


7 


Entry  Time 

The  results  of  a four-way  analysis  of  covariance  performed  on  the 
time  taken  per  entry  are  presented  in  Table  3.  The  four  factors  ana- 
lyzed were  type  of  training  feedback,  type  of  transfer-testing  feed- 
back, type  of  message  (enemy  versus  friendly) , and  sessions  (training 
versus  transfer-testing) . The  data  from  18  participants  were  excluded 
from  all  of  the  data  analyses  because  six  participants  failed  to  com- 
plete the  first  31  transfer-testing  messages,  one  failed  to  complete 
training,  one  became  ill,  one  failed  to  return  on  the  second  day,  and 
nine  were  eliminated  due  to  system  malfunctions.  Table  2 shows  the 
number  of  participants  remaining  in  each  of  the  training/transfer  feed- 
back combinations.  A chi  squared  test  was  applied  to  the  number  of  par- 
ticipants eliminated  from  each  of  the  groups  to  determine  the  likelihood 
that  the  loss  of  data  from  any  of  the  groups  was  due  to  a systematic 
bias.  This  test  showed  that  the  probability  of  such  a systematic  bias 
was  low  (x2(2)  = 2.26,  p > .25). 

Data  from  all  21  training  messages  were  used  whenever  training  was 
analyzed.  Only  the  data  from  the  first  31  transfer-testing  messages 
were  selected  for  analysis  because  (a)  most  participants  failed  to  com- 
plete all  43  possible  messages,  and  (b)  the  first  31  messages  included 
participant  responses  to  all  10  possible  formats. 

Training.  The  mean  entry  times  for  type  of  feedback  given  and 
type  of  message  received  during  training  are  presented  in  Table  4. 

The  results  show  unequivocally  the  potential  time-saving  characteristic 
of  the  response-sensitive  instructional  strategy  used  here.  During 
training,  Response-Sensitive  Feedback  produced  a mean  savings  of  4.09 
seconds  per  item  over  the  Combined  Control  Feedback  conditions  on  Enemy 
messages  (p  < .01) . Response-Sensitive  Feedback  also  produced  signifi- 
cant time  savings  on  Enemy  training  messages  when  compared  to  Remedial 
Feedback;  the  mean  time  per  entry  was  3.82  seconds  faster  under  Response- 
Sensitive  Feedback  (p  < .01) . Remedial  and  Combined  Control  Feedback 
conditions  did  not  differ  significantly  from  one  another  on  Enemy 
training  messages. 

Because  auto-fill  could  not  occur  on  Friendly  messages,  it  is 
not  surprising  that  the  Response-Sensitive  Feedback  condition  did  not 
differ  significantly  from  the  Remedial  Feedback  condition  on  mean  item 
entry  times.  Remedial  Feedback  produced  significantly  faster  entry 
times  than  did  Combined  Control  Feedback  on  Friendly  training  messages 
(p  < .05) , but  Response-Sensitive  Feedback  did  not.  This  result  was 
unexpected,  because  the  Response-Sensitive  and  Remedial  Feedback  groups 
received  identical  feedback  on  Friendly  messages.  Failure  of  Response- 
Sensitive  Feedback  to  produce  faster  entry  times  on  Friendly  messages 
may  have  been  due  to  the  differences  in  feedback  experienced  by  this 
group  between  Friendly  messages  without  auto-fill  and  Enemy  messages, 
on  which  auto-fill  was  available.  The  fact  that  Enemy  messages  were 
processed  significantly  faster  than  Friendly  messages  (X  = 10.04 
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Table  3 


Analysis  of  Covariance  for  Mean  Time  per  Entry 


Source 

df 

MS 

F 

Between  subjects 

Training  Feedback  (TF) 

2 

19.59 

<1 

Transfer-Testing  Feedback  (TTF) 

1 

6.11 

<1 

TF  x TTF 

2 

34.84 

1.58 

Errorb 

46 

22.06 

Within  subjects 

Message 

Type  (M) 

1 

82.34 

15.45*** 

M x TF 

2 

37.85 

7.10** 

M x TTF 

1 

.45 

<1 

M x TF  : 

x TTF 

2 

20.48 

3.84* 

Error 3 

47 

5.33 

Sessions  (S) 

1 

1135.62 

121.58*** 

S x TF 

2 

50.62 

4.29* 

S X TTF 

1 

.01 

<1 

S x TF  : 

K TTF 

2 

10.69 

<1 

Error2 

47 

11.81 

M x S 

1 

244.31 

60.52*** 

M x S x 

TF 

2 

46.85 

11.61*** 

M x S x 

TTF 

1 

.01 

<1 

M x S x 

TF  x TTF 

2 

8.24 

2.04 

Error3 

47 

4.04 

*p  < .05. 

**p  < .01. 

***p  < .001. 


sec/item  versus  X = 16.59  sec/item)  in  the  Response-Sensitive  group 
(F ( 1 , 94)  = 82.67,  p < .001)  lends  support  to  this  hypothesis.  There 
were  no  significant  differences  in  entry  times  between  Enemy  and 
Friendly  messages  for  any  other  feedback  group. 

Table  4 

Mean  Item  Entry  Time  in  Seconds  for  Type  of 
Message  and  Mode  of  Training 


Training  Mode 

a 

n 

Me  s sac 
Enemy 

je  type 
Friendly 

Training  Session 

Combined  Control 

16 

14.13 

17.47 

Remedial  Feedback 

19 

13.86 

14.16 

Response-Sensitive  Feedback 

18 

10.04 

16.59 

Transfer-Testing  Session 

Combined  Control 

16 

9.12 

8.43 

Remedial  Feedback 

19 

9.55 

8.65 

Response-Sensitive  Feedback 

18 

10.15 

8.96 

aNumber  of  scores  combined  across  transfer-testing  modes  and  participants. 


The  effect  of  Response-Sensitive  Feedback  on  training  time  can  be 
summarized  as  follows.  Compared  to  the  Combined  Control  group,  Response- 
Sensitive  Feedback  produced  a mean  time  savings  of  40  minutes  over  the 
course  of  the  21  training  messages.  It  also  resulted  in  a mean  savings 
of  37  minutes  over  Remedial  Feedback  training. 

Transfer-Testing.  The  time  savings  produced  by  Response-Sensitive 
Feedback  training  did  not  transfer  to  the  simulated  operational  environ- 
ment. In  fact,  no  significant  differential  effects  on  transfer-testing 
entry  times  were  produced  by  any  of  the  training  feedback  conditions. 

As  Table  4 shows,  all  participants  made  their  entries  at  about  the  same 
rate  during  transfer-testing,  and  there  were  no  significant  differences 
in  entry  times  between  Enemy  and  Friendly  messages. 


These  results  contradict  those  of  Strub  (1975) , who  found  faster 
inputting  speeds  for  Friendly  messages  under  all  conditions.  However, 
Strub  used  mean  time  in  minutes  per  format,  whereas  we  used  time  in 
seconds  per  entry  as  the  measure  of  input  entry  speed.  Use  of  the  entry 
time  per  format  measurement  necessarily  makes  input  speed  appear  slower 
for  Enemy  messages  since  the  Enemy  format  contains  on  the  average  12 
more  entries  than  the  Friendly  formats.  If  one  applies  the  time-per- 
entry  analysis  to  Strub' s data,  the  differences  in  entry  speed  between 
Enemy  and  Friendly  messages  virtually  disappear,  with  Enemy  message 
entry  times  averaging  slightly  over  1.5  seconds  slower  than  Friendly 
message  entry  times. 

It  should  be  noted  that  type  of  transfer-testing  feedback  did  not 
influence  response  times.  The  error  detection  routines  used  in  the 
Edit  Feedback  condition  did  not  significantly  affect  inputting  speed. 


Entry  Accuracy 

First-Entry  Accuracy.  A four-way  analysis  of  covariance  was  used 
to  assess  the  effects  of  training  and  transfer-testing  feedback  condi- 
tions on  accuracy  of  performance. 4 The  results  of  this  analysis  appear 
in  Table  5.  The  four  factors  analyzed  were  type  of  training  feedback, 
type  of  transfer-testing  feedback,  type  of  message,  and  sessions. 

Because  a response-sensitive  training  strategy  enables  students 
to  bypass  material  already  learned  and  to  concentrate  their  efforts 
on  items  not  yet  learned,  it  was  assumed  that  the  use  of  this  strategy 
would  facilitate  the  transfer  of  MIOD  training  to  operational  testing 
environments.  Table  6 shows  that  the  results  of  the  present  experiment 
do  not  support  this  assumption.  Only  the  Edit  Feedback  trained  and 
tested  participants  showed  even  a hint  of  positive  transfer  of  train- 
ing as  evidenced  by  a small,  nonsignificant  increase  in  performance 
accuracy. 


An  unresolved  error  in  the  computer  program  used  for  automatically 
scoring  participant  entries  produced  some  falsely  incorrect  entries 
and  generated  several  instances  of  false  error  feedback  during  train- 
ing. The  incidence  of  such  errors  was  small,  and  the  effects  on  the 
performance  of  the  participants  involved  was  judged  to  be  minimal. 
Specifically,  there  were  63  such  cases  of  computer-generated  error  in 
the  44,679  entries  made  during  training  (.14%).  Prior  to  computing  the 
accuracy  scores  for  each  participant,  however,  all  entry  errors  were 
manually  rescored  and  corrected  for  computer  error. 
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Table  5 

Analysis  of  Covariance  for  First  Entry  Accuracy 


Source 

df 

MS 

F 

Between  subjects 


Training  Feedback  (TF) 

2 

1294.27 

9.39* ** 

Transfer-Testing  Feedback  (TTF) 

1 

399.49 

2.90 

TF  x TTF 

2 

140.42 

1.02 

Error ^ 

46 

137.85 

Within  subjects 

Message 

Type  (M) 

1 

1165.84 

60.40** 

M x TF 

2 

159.77 

8.28** 

M X TTF 

1 

39.76 

2.06 

M x TF  x 

: TTF 

2 

4.22 

<1 

Error^ 

47 

19.30 

Sessions  (S) 

1 

730.20 

51.17** 

S x TF 

2 

102.85 

7.21** 

S x TTF 

1 

.17 

<1 

S X TF  X 

: TTF 

2 

83.34 

5.84** 

Error 2 

47 

14.27 

M x S 

1 

1.60 

<1 

M x S x 

TF 

2 

12.15 

<1 

M x S x 

TTF 

1 

137.90 

7.77** 

M x S x 

TF  x TTF 

2 

13.19 

<1 

Error 3 

47 

17.75 

*t  < .01. 

**p  < .001. 
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Table  6 


Mean  Percentage  Accuracy  Scores  for  Training  and  Transfer-Testing 
Sessions  as  a Function  of  Training  and 
Transfer-Testing  Feedback  Modes 


Training  Mode 

a 

n 

Training 

Session 

Transfer-Testing 

No  Feedback 

Transfer-Testing  Mode 

Minimum  Feedback13 

14 

79.36 

75.04 

Remedial  Feedback 

20 

89.35 

87.10 

Response-Sensitive 

Feedback 

16 

86.41 

81.72 

Edit  Feedback 

Transfer- 

-Testing  Mode 

Edit  Feedback 

18 

81.72 

82.61 

Remedial  Feedback 

18 

89.96 

85.86 

Response-Sensitive 

Feedback 

20 

91.94 

83.76 

a 

Number  of  scores  combined  across  message  types  and  participants. 


Control  condition. 


Under  No  Feedback  testing  conditions,  both  Minimum  and  Response- 
Sensitive  Feedback  training  groups  showed  a significant  decline  in  per- 
formance during  transfer-testing  (F(l,  47)  * 12.33,  p < .01).  The  de- 
cline in  performance  by  the  Remedial  Feedback  training  group  was  not 
significant. 

Under  Edit  Feedback  testing,  both  the  Remedial  and  Response- 
Sensitive  Feedback  training  groups  showed  a significant  decline  in 
performance  during  transfer-testing  (F(l,  47)  * 10.60,  p < .01  and 
F { 1 , 47)  =■=  46.89,  p < .01,  respectively).  The  Edit  Feedback  trained 
and  tested  participants  showed  a small,  if  nonsignificant,  gain  in 
performance  accuracy  during  transfer-testing.  This  result  is  not  sur- 
prising, since  training  and  transfer-testing  feedback  conditions  were 
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identical,  and  one  would  expect  positive  (or  at  least  nonnegative) 
transfer  to  occur. 


Table  6 also  shows  that  neither  Remedial  Feedback  training  nor 
Response-Sensitive  Feedback  training  provided  participants  with  any 
special  advantage  when  tested  under  the  Edit  Feedback  compared  to  the 
No  Feedback  condition,  the  differences  in  performance  in  each  case  be- 
ing trivial  and  nonsignificant.  The  Edit  trained  and  tested  partici- 
pants were  clearly  superior  in  performance  to  those  receiving  Minimum 
Feedback  training  and  No  Feedback  transfer-testing  (F(l,  93)  = 5.68, 
p < .05).  This  result  was  probably  due  to  the  slight  positive  transfer 
found  in  the  Edit  trained  and  tested  participants  combined  with  the 
general  decrement  in  transfer-testing  performance  experienced  by  all 
other  participants.  The  accuracy  scores  in  Table  6 show  that  there 
were  no  significant  differences  in  training  performance  between  No 
Feedback  and  Edit  Feedback  transfer-testing  conditions  for  any  of  the 
training  groups.  This  finding  is  important  for  two  reasons.  First, 
the  lack  of  differences  between  the  two  Remedial  Feedback  training 
groups  (F ( 1 , 93)  < 1,  p > .05)  and  between  the  two  Response-Sensitive 
training  groups  (F(l,  93)  = 3.42,  p > .05)  indicates  that  no  systematic 
confounding  occurred  in  the  initial  assignment  of  participants  to  these 
groups.  Secorid,  the  absence  of  differences  between  the  two  control 
groups  (F ( 1 , 93)  < 1,  p > .05)  during  training  suggests  that  Minimum 
Feedback  and  Edit  Feedback  are  similar  in  their  influence  on  response 
accuracy. 

Since  transfer-testing  feedback  conditions  were  not  in  effect 
during  training  and  since  the  Minimum  Feedback  and  Edit  Feedback  groups 
did  not  differ  significantly  during  training,  the  transfer-testing 
factor  was  ignored  in  analyzing  the  effect  of  type  of  training  feed- 
back on  performance  accuracy  during  the  training  session.  Minimum 
Feedback  and  Edit  Feedback  groups  were  considered  to  be  a single  group 
(i.e.,  Combined  Control)  with  an  overall  mean  of  80.69.  The  mean  per- 
formance of  Remedial  and  Response-Sensitive  Feedback  groups  was  virtu- 
ally the  same  during  training  (89.64  and  89.48,  respectively).  Both 
groups  show  significant’ / superior  performance  compared  to  the  Combined 
Control  group  when  tested  with  Tukey's  HSD  test  (p  < .01  in  both  cases). 

During  transfer-testing , the  performance  accuracy  for  Remedial  and 
Response-Sensitive  Feedback  trained  groups  was  also  higher  than  their 
respective  control  groups  under  both  No  Feedback  and  Edit  Feedback 
testing  conditions.  However,  the  only  statistically  significant  dif- 
ference was  for  the  Remedial  Feedback  trained  group  compared  to  the 
Minimum  Feedback  trained  group  under  No  Feedback  testinq  (p  .01) . 
Although  Remedial  and  Response-Sensitive  training  strategies  were  mildly 
effective  in  improving  performance  accuracy  during  training,  little  of 
this  positive  influence  was  transferred  to  either  of  the  two  testing 
environments  used  here. 


i 
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Strub  (1975)  found  that  participants  completed  Friendly  message 
formats  with  greater  accuracy  than  Enemy  message  formats.  Our  results 
agree  with  Strub' s in  that  the  overall  accuracy  on  Friendly  messages 
(87.32%)  was  significantly  higher  than  on  Enemy  (82.65%)  messages 
(F(l,  47)  = 60.40,  p < .001).  The  analysis  of  covariance  revealed  a 
significant  training  feedback  x message  type  interaction  (F(2,  47)  = 
8.28,  p < .001) . This  result,  coupled  with  the  fact  that  the  training 
feedback  x message  type  x sessions  interaction  was  not  significant, 
indicates  that  performance  accuracy  on  Enemy  and  Friendly  messages  was 
differentially  affected  by  the  various  training  feedback  conditions. 
Furthermore,  this  effect  was  essentially  the  same  during  transfer  test- 
ing as  it  was  during  training.  Table  7 shows  the  mean  accuracy  scores 
on  Enemy  and  Friendly  messages  for  each  of  the  three  training  feedback 
conditions.  Subsequent  analyses  showed  that  all  three  training  feedback 
groups  performed  more  accurately  on  Friendly  than  on  Enemy  messages 
(F ( 1 , 47)  = 57.06,  p < .001;  F(l,  47)  * 17.32,  p < .001;  and  F(l,  47)  = 
7.59,  p < .001  for  Combined  Control,  Remedial  Feedback,  and  Response- 
Sensitive  groups,  respectively) . The  results  of  HSD  tests  revealed 
that  on  Enemy  messages  both  Remedial  and  Response-Sensitive  groups  per- 
formed significantly  better  than  the  Combined  Control  group  (p  < .001 
in  both  cases) , but  did  not  differ  significantly  from  each  other  in 
performance  accuracy.  On  Friendly  messages,  both  Remedial  and  Response- 
Sensitive  groups  performed  more  accurately  than  the  Combined  Control 
group,  although  the  difference  was  significant  only  for  the  Remedial 
group  (p  < .05).  As  on  the  Enemy  messages,  the  two  groups  did  not  dif- 
fer significantly  from  each  other  in  performance  accuracy.  Remedial 
and  Response-Sensitive  Feedback  training  led  to  reduced  errors  on  both 
Enemy  and  Friendly  messages.  However,  Remedial  Feedback  was  somewhat 
more  effective  than  Response-Sensitive  Feedback  in  eliminating  errors 
on  Friendly  messages.  This  result  must  be  interpreted  cautiously  since 
auto-fill  was  not  given  on  Friendly  messages. 


Table  7 

Mean  Percentage  Accuracy  Scores  for  Type  of  Message 
and  Mode  of  Training 


Message  type 


Training  Mode 

d 

n 

Enemy 

Friendly 

Combined  Control 

32 

75.84 

84.14 

Remedial  Feedback 

38 

85.98 

90.17 

Response-Sensitive  Feedback 

36 

85.20 

87.13 

UNumber  of  scores  combined  across  transfer-testing  conditions,  sessions, 
and  participants. 
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As  can  be  seen  in  Table  8,  the  type  of  transfer-testing  feedback 
also  affected  performance  on  Enemy  and  Friendly  messages  differently. 
Under  Edit  Feedback,  performance  was  significantly  more  accurate  on 
Friendly  messages  than  on  Enemy  messages  (F(l,  94)  = 36.97,  p < .001). 
Performance  did  not  differ  under  the  No  Feedback  condition.  These  dif- 
ferences are  primarily  due  to  the  lowered  performance  on  Friendly  mes- 
sages found  under  No  Feedback  testing  as  compared  to  Edit  Feedback 
testing.  However,  this  difference  was  not  statistically  significant. 


Table  8 

Mean  Percentage  Testing  Accuracy  Scores  for  Type  of  Message 
and  Type  of  Transfer-Testing  Feedback 


a 

n 

Message 

type 

Transfer-Testing  Feedback 

Enemy 

Friendly 

No  Feedback 

25 

81.03 

82.96 

Edit  Feedback 

28 

80.57 

87.56 

a 


Number  of  scores  combined  across  training  feedback  modes  and  participants. 


First  Versus  Final  Entry  Accuracy.  A four-way  analysis  of  covari- 
ance was  used  to  assess  the  relative  effectiveness  of  the  various  train- 
ing feedback  conditions  in  reducing  detectable  and  nondetectable  error 
rates.  The  results  are  presented  in  Table  9.  The  four  factors  analyzed 
were  type  of  training  feedback,  type  of  transfer-testing  feedback,  type 
of  message,  and  error  detection  accuracy.  Error  detection  accuracy  was 
measured  by  comparing  first  and  last  entry  accuracy  during  transfer- 
testing. Differences  between  first  and  last  entry  were  observed  only 
in  those  participants  receiving  Edit  Feedback  during  transfer-testing, 
since  they  alone  were  allowed  to  correct  illegal  entries  made  on  first 
entries.  Most  of  the  errors  made  in  the  testing  situation  were  not 
caught  by  the  Edit  routine.  Although  the  difference  in  performance  ac- 
curacy between  the  first  entry  and  the  entry  finally  accepted  by  the 
computer  was  significant  (F(l,  47)  = 69.48,  p < .001),  it  can  hardly 
be  called  important  (mean  accuracy  = 84.12%  and  85.21%  for  first  and 
final  entries,  respectively).  This  performance  difference  of  1.09% 
means  that  only  7%  of  all  errors  were  detected  by  the  Edit  routine. 

The  lack  of  a significant  error  detection  x accuracy  training  feedback 
interaction  (F(2,  47)  < 1,  p < .05)  shows  that  all  of  the  training  feed- 
back conditions  were  equally  effective  (ineffective?)  in  reducing  final 
error  rates.  Since  none  of  the  interactions  involving  either  message 
type  and/or  error  detection  accuracy  were  significant,  it  appears  that 


the  error  detection  routine  was  equally  effective  on  Enemy  and  Friendly 
messages. 


Analysis  of  Covariance  for  First  Versus  Last  Attempt  Accuracy 


Source 

df 

MS 

F 

Between  subjects 

Training  Feedback  (TF) 

2 

1022.37 

6.33* 

Transfer-Testing  Feedback  (TTr) 

1 

588.33 

3.65 

TF  x TTF 

2 

335.45 

2.08 

Errorb 

46 

161.39 

Within  subjects 

Attempts  (A) 

1 

15.61 

69.48** 

A x TF 

2 

.11 

<1 

A x TTF 

1 

15.61 

69.48** 

A x TF  x TTF 

2 

.11 

<1 

Error ^ 

47 

.22 

Message  Type  (M) 

1 

1112.43 

29.67** 

M x TF 

2 

101.68 

2.71 

M x TTF 

1 

343.09 

9.15* 

M x TF  x TTF 

2 

7.66 

<1 

Error2 

47 

37.49 

A x M 

1 

.22 

3.58 

A x M x TF 

2 

.14 

2.24 

A x M x TTF 

1 

.22 

3.58 

A x M x TF  x TTF 

2 

.14 

2.24 

Error 3 

47 

.06 

*p  < .01. 

**p  < .001. 
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SUMMARY  AND  CONCLUSIONS 


In  summary,  a response-sensitive  training  technique,  covering  only 
those  format  items  in  which  errors  occur,  is  effective  in  reducing 
training  time.  This  technique  is  also  at  least  as  effective  as  other 
training  strategies  in  reducing  error  rates.  However,  the  decision  to 
adopt  a response-sensitive  strategy  for  a specific  training  problem 
should  consider  the  trade-off  between  the  cost  of  training  development 
and  the  benefit  of  reduced  training  time. 

The  type  of  error  detection  routine  provided  by  Edit  Feedback  was 
only  minimally  effective  in  reducing  errors  and  was  not  enhanced  by  any 
of  the  training  methods  used.  The  results  of  the  present  experiment 
suggest  that  training  probably  will  not  help  catch  many  of  the  errors 
that  escape  the  usual  operational  error  detection  routines.  Although 
more  sophisticated  error-detection  routines  might  help  reduce  such  er- 
rors, a simpler  and  perhaps  more  effective  way  to  eliminate  the  errors 
would  be  to  clarify  the  meaning  of  the  terms  used  as  legal  entries  in 
the  various  formats.  For  example,  there  was  confusion  among  participants 
about  the  terms  Observation  and  Reconnaissance  as  descriptors  for  in- 
formation sources.  The  distinction  between  other  items,  such  as  "Mines"  ' 

versus  "Demolition  Equipment"  and  "Demolished"  versus  "Destroyed,”  was 
also  unclear.  If  these  terms  were  changed  or  clarified,  a fairly  high 
percentage  of  otherwise  undetectable  and  unavoidable  errors  might  be 
eliminated. 
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APPENDIX 

DESCRIPTION  OF  DATA  ENTRY  FORMATS 

Table  A-l 
Format  Index 


Designator 


Description 


ECla 

UC1 

UD1A 

UE1 

UF1 

UG1A 

UG1B 

UG1C 

UG1F 

UG1I 


Enemy  situation/action 
Front  line  trace/coordination  point 
Command  post/center  of  mass 
General  on  post/combat  on  post 
Assembly  area/operational  area 
Line  of  departure/phase  lines 
Axis  of  advance/direction  of  attack 
Objectives 

Movement  control  points 
Fire  control  lines 


a 

All  information  about  enemy  activity  was  entered  on  the  EC1  format. 
The  remaining  nine  formats  were  used  to  describe  friendly  activities 
only. 
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Table  A-2 


Training  Formats 


Message  number 

Format  required 

Message  number 

Format  required 

1 

UC1 

33 

UG1F 

2 

UE1 

34 

UF1 

3 

UE1 

35 

EC1 

4 

UE1 

36 

II  It 

5 

UD1A 

37 

II  II 

6 

UG1A 

38 

II  II 

7 

EC1 

39 

II  II 

8 

II  II 

40 

UEl 

9 

II  II 

41 

II  II 

10 

II  II 

42 

EC1 

11 

II  II 

43 

II  II 

12 

II  II 

44 

II  II 

13 

II  II 

45 

II  II 

14 

II  II 

46 

II  II 

15 

•I  II 

47 

II  II 

16 

II  II 

48 

•I  II 

17 

•1  II 

49 

II  II 

18 

•1  II 

50 

UG1C 

19 

UG1C 

51 

UG1A 

20 

UG1A 

52a 

UG1B 

21 

UG1B 

53 

UC1 

22 

UG1I 

54 

II  II 

23 

UC1 

55 

II  II 

24 

UG1I 

56 

UD1A 

25 

II  II 

57 

EC1 

26 

UD1A 

58 

II  II 

27 

UEl 

59 

II  II 

28 

•1  II 

60 

II  II 

29 

II  II 

61 

It  II 

30 

EC1 

62 

II  II 

31 

II  II 

63 

II  II 

32 

II  It 

64 

II  II 

a 

Participant  responses  to  items  beyond  message  number  52  were  not  used 
in  data  analysis. 


Table  A-3 

Format  Scorable  Entries 


Format  designator 

Number  of  entries 

EC1 

45 

UC1 

24 

UD1A 

21 

UE1 

25 

UF1 

31 

UG1A 

46 

UG1B 

47 

UG1C 

59 

UG1F 

51 

UG1I 

46 

Data  Entry  Format — Printouts 


ECl  ENSIT  DATA  MtSSAGE 


nSG-NH  ( 

) SC  T Y ( ) OMlG ( ) RFSTH ( 

RtCD-F  H ( 

>KPT-IO< 

) 

AGFNCY ( 

) SOUHCfc  ( 

/ 

) 

r VFNT-TiMk  ( 

>Em-UN1T<  / 

/ ) 

— SUM  J AC  I V— UUANT 

-DESCRIPTION 

— LOCATION 

1 ( / 

/ 

/ 

/ 

<M  / 

/ 

/ 

/ 

*<  / 

/ 

/ 

/ 

A ( / 

/ 

/ 

/ 

S(  / 

/ 

/ 

/ 

PAW!  NT  < / 

HtMAMKS ( 

/ 

)HFLAT-TO(  > 

I>IS1M(  t 

• t 

. ) INTEREST 

( ) 
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THIS  PAGE  IS  BEST  QUALITY  PRACTICABLE 
FROM  QQPX  jf^RMISHSD  TO  DDC  


Data  Entry  Format — Printouts  (continued) 


. 1C  1 FLT  COOMU  UAIA  MESSAGE 

ib(,-NR<  )PHEL(  )SCTY(  )0HIG<  ) RFSTH  ( ) 

unit-io-oh-if-namu  / / / / ) 

(•LT-COOWO-i  < )FLT-f  OOHU-2(  ) 

t-LT-fOOHu-J  ( )f-LT-rO0RD-4  < ) 

FLT-T  IMF  ( ) 

CO-ON-LlNt-POSIT ( / ) 

HN-ON-LItMt-POSIT  ( / ) 

LT-COOMO-Pf ( ) 

L T-COOHD-PT-TIME ( ) 

HT-C00RU-PF<  ) 

WT-COOHU-Pl-TIME ( ) 


OU1A  CP-COM  UATA  MfcSSAGt 


XSG-NR ( 

) PRtC ( ) Sd  Y ( 

) OR IG ( 

>RFSTR ( 

) 

UNJT-IU-OR- 

T F-NAMt ( / / 

/ / 

> 

— HfiWLOCATION/ TIME- 

— / 

-CuK-CP< 

/ 

) 

-Pl.N-CP  ( 

/ 

) 

-A|_T-CP< 

/ 

) 

CUR-COM ( 

/ 

) 

PLN-rOM( 

/ 

) 

Ufc I GOP-COP 

DATA  MESSAGE. 

mSg-nR ( 

)PREC(  )SCTY( 

) OR IG ( 

>pfstr< 

) 

UNIT-IO-UR- 

(F-NAMt ( / / 

/ / 

) 

l»OP-NAMt  ( 

) 

LOr  ( 

• * 

* 

) 

T I -FR  ( 

) TIME- 

TO  ( 

) 

COP-NAME  ( 

) 

| 

LOC( 

* • 

t 

) 

T I MF  -F  « ( 

) time- 

TO  ( 

) 

IMIS  PASS  IS  BEST  QUALITY  PRACTICABLE 
JSW1  QQFY  PURlMT.^^n  rnnw 


Data  Entry  Printouts  (continued) 


UOIH  CON  Mb AS  DATA  message  AXIS  AOV  ♦ OIR  attack 
mSG-NW(  ) PREC  ( )SCTY(  )0RIG(  ) BFSTR  ( ) 


UNI T-lO-OR-IF-NAMt(  / / / / ) 

AX TS-OF-AUV-l-NAMt ( ) 

l.Of;  ( * » * 

AXIS-OF-AUV-£>-NAMt(  ) 

LOC(  « « « 

/ X I S-OF-aUV-J-NAMt ( ) 

LOC(  * . * 

-UIH-OF-mTK-} -NAML ( ) 

LOC ( « » « 

-OIR-OF-ATK-i>-NAMt(  ) 

LOC ( » . * 

-OIR-OF-AlK-J-NAMt ( ) 

LOC.  ( « « t 


I 


OTIS  PAGE  IS  BEST  QUALITY  PRACTICABLE 

FROM  COPY  FURNISHED  TO  DDC  - ' 

Data  Entry  Format — Printouts  (continued) 

UblF  CON  Mt AS  DATA  MESSAGF  MUT  CTL  POINTS 
mSg-N«(  ) PRfcL  ( )SCTY(  ) ON IG ( ) PFSTH ( ) 

• »NlT-IO-OH-TF-NAMt(  / / / / ) 


-1  TfcM-i 

NN-LOCAT ION 

i—  ITFJM- 

np-location 

-SP-)  ( 

/ 

) -SP-2 ( 

/ 

-SP-1 ( 

/ 

)-SP-M 

/ 

-NP-1  ( 

/ 

) ”RP-2  ( 

/ 

-HP-1 ( 

/ 

) -HP-4  ( 

/ 

LUP-1  ( 

/ 

) LUP-2 ( 

/ 

LOP-1 ( 

/ 

) LUP-4 ( 

/ 

CKP-I  ( 

/ 

)CKP-<i( 

/ 

CKP- * ( 

/ 

) CKP-4  ( 

/ 

-PP-1  ( 

/ 

>-PP-iM 

/ 

-PP-1 ( 

/ 

)-pp-4( 

/ 

UGl  I CON  Mt  AS  UAI A 

MESSAGE  FIRE 

CONTPOI. 

LINES 

'*SG— NH  ( 

)PHEC( 

>SCTY(  )OHIG< 

) PFSTR ( ) 

IIN1T-IU-OH- 

fF-NAMt ( 

/ / / / 

) 

FSCL-LOC ( 

t 

• 

♦ 

t 

t 

>TlME< 

) 

-NFI.-LOC  ( 

* 

t 

f 

t •*••• 

f 

) TIME  ( 

) 

-FCL-LOC ( 

t 

• 

t 

t •*••• 

t 

) T IME ( 

) 

nHH-NAME ( 

)l  OC( 

t 

i — - 

t 

• 

t 

) 

HHO-NAMt ( 

) I oc  ( 

« 

, ... 

« 

• 

* 

) 

27 
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1 HQUSMC.  Commandant.  ATTN:  Corle  MPI  20 

2 USCG  Academy,  New  London.  ATTN:  Admission 
2 USCG  Academy,  New  London,  ATTN:  Library 

1 USCG  Training  Ctr,  NY,  ATTN:  CO 
1 USCG  Training  Ctr.  NY.  ATTN:  Educ  Svc  Ofc 
1 USCG,  Psychol  Res  Br,  DC,  ATTN;  GP  1/62 
1 HQ  Mid-Range  Br,  MC  Det,  Quantico,  ATTN:  P&S  Div 


1  US  Marine  Corps  Liaision  Ofc,  AMC.  Alexandria,  ATTN:  AMCGS— F 
1 USATRADOC.  Ft  Monroe,  ATTN:  ATRO-ED 

6 USATRADOC.  Ft  Monrot.  ATTN:  ATPR-AD 
1 USATRADOC.  Ft  Monrot,  ATTN:  ATTS-E A 

1 USA  Forets  Cmd,  Ft  McPherson,  ATTN:  Library 

2 USA  Aviation  Test  Bd,  Ft  Rucker,  ATTN:  STEBG— PO 

1 USA  Agey  for  Aviation  Safety,  Ft  Rucker,  ATTN:  Library 
1 USA  Agey  for  Aviation  Safety.  Ft  Rucker,  ATTN:  Educ  Advisor 
1 USA  Aviation  Sch.  Ft  Rucker,  ATTN:  PO  Drawer  0 

1 HQUSA  Aviation  Sys  Cmd.  St  Louis.  ATTN:  AMSAV-ZDR 

2 USA  Aviation  Sys  Twt  Act..  Edwards  AFB,  ATTN:  SAVTE-T 
t USA  Air  Def  Sch.  Ft  Bliss,  ATTN:  ATSA  TEM 

1 USA  Air  Mobility  Rsch  & Dev  Lab,  Moffett  Fid,  ATTN:  SAVDL-AS 
1 USA  Aviation  Sch.  Res  Tng  Mgt,  Ft  Rucker,  ATTN:  ATST— T-RTM 
1 USA  Aviation  Sch,  CO.  Ft  Rucker,  ATTN:  ATST— D-A 
1 HQ.  DARCOM.  Alexandria,  ATTN:  AMXCD-TL 
1 HQ.  DARCOM.  Alexandria,  ATTN:  CDR 
1 US  Military  Academy,  West  Point.  ATTN:  Serials  Unit 
1 US  Military  Academy,  West  Point,  ATTN:  Ofc  of  Milt  Ldrshp 
1 US  Military  Academy,  West  Point,  ATTN:  MAOR 
1 USA  Standardization  Gp,  UK.  FPO  NY.  ATTN:  MASE-GC 
1 Ole  of  Naval  Rsch,  Arlington,  ATTN:  Code  452 

3 Ofc  of  Naval  Rsch,  Arlington,  ATTN:  Code  468 
1 Ofc  of  Naval  Rsch,  Arlington,  ATTN:  Code  450 
1 Ofc  of  Naval  Rsch,  Arlington,  ATTN:  Code  441 

1 Naval  Aerospc  Med  Res  Lab,  Pensacola,  ATTN:  Acous  Sch  Div 
1 Naval  Aerospc  Med  Res  Lab,  Pensacola,  ATTN:  Code  L51 
1 Naval  Aerospc  Med  Res  Lab,  Pensacola,  ATTN:  Code  L5 
1 Chief  of  NavPers,  ATTN:  Pers-OR 
1 NAVAI RST A.  Norfolk.  ATTN:  Safety  Ctr 
1 Nav  Oceanographic,  DC,  ATTN:  Code  6251,  Charts  & Tech 
1 Center  of  Naval  Anal,  ATTN:  Doc  Ctr 
1 NavAirSysCom,  ATTN:  AIR-531 3C 
1 Nav  BuMed.  ATTN:  713 
1 NavHelicopterSubSqua  2,  FPO  SF  96601 
1 AFHRL(FT)  William  AFB 
1 AFHRL  (TT)  Lowry  AFB 

1 AFHRL  (AS)  WPAFB.OH 

2 AFHRL  (DOJZ)  Brooks  AFB 

1 AFHRL  (DOJN)  Lackland  AFB 
1 HQUSAF  (INYSD) 

1 HQUSAF (DPXXA) 

1 AFVTG  (RD)  Randolph  AFB 

3 AMRL  (HE)  WPAFB,  OH 

2 AF  Inst  of  Tech,  WPAFB.  OH.  ATTN:  ENE/SL 
1 ATC  (XPTD)  Randolph  AFB 

1 USAF  AeroMcil  Lib.  Brooks  AFB  (SUL  4).  ATTN:  DOC  SEC 
1 AFOSR  (NL),  Arlington 

1 AF  Log  Cmd.  McClellan  AFB.  ATTN:  ALC/DPCR0 

1 Air  Force  Academy,  CO,  ATTN:  Dept  of  Bel  Sen 
5 NitvPors  ft  Dev  Ctr,  San  Diego 

2 Navy  Med  Neuropsychiatric  Rsch  Unit.  San  Diego 
1 N.iv  Electronic  Lab.  San  Diego,  ATTN:  Res  Lab 

1 Nav  TrngCen,  San  Diego,  ATTN:  Code  0000- Lib 
1 NnvPostGrnSch,  Monterey,  ATTN:  Code  55Aa 
1 NavPostGiaSch,  Monterey,  ATTN:  Code  2124 
1 Nav  TrntjE quipCtr,  Orlando,  ATTN:  Tech  Lib 
1 US  Dept  ol  Lalxir,  DC,  ATTN:  Manpower  Admin 
1 US  Dept  of  Justice,  DC,  ATTN:  Drug  Enforce  Admin 
1 Nat  Bur  of  Standards.  DC.  ATTN:  Computer  Info  Section 
1 Nut  Clearing  Hut  iso  for  MH-Info,  Rockville 
1 Denver  l ednuil  Ctr.  Laknwood,  ATTN:  BLM 
1?  Defense  Documentation  Center 

4 Dir  Psych,  Army  Hrj.  Russell  Ofcs,  Canberra 

I Scientific  Adir.i,  Mil  Bd,  Army  Hg,  Russell  Ofcs.  Canberra 
1 Mil  .ind  An  Attache.  Austrian  Embassy 
1 Cellin'  de  Reel. niche  |)es  Fat:  tours.  Humaine  de  la  Defense 
Nution.ile,  Hiuvu-ls 

7 Cho.mIi.io  Joint  Stall  W.rhmyton 

I C/Am  Stall,  Royal  Conadian  AT.  ATTN:  ParsStd  Anti  Br 
.1  Chief,  Canadian  Del  Rsch  Staff.  ATTN:  C/CRDS(W) 

4  Riitish  iN’f  Staff.  British  Embassy,  Washington 


1 Def  ft  Civil  Inst  of  Enviro  Medicine,  Canada 
1 AIR  CRESS.  Kensington,  ATTN:  Info  Sys  Br 
1 Militatrpaykologitk  Tjeneste,  Copehagen 
1 Military  Attacha,  French  Embassy,  ATTN:  Doc  Sac 
1 Madacin  Chaf.  C.E.R.P.A.-Artanal.  Toulon /Naval  Franca 
1 Prin  Scientific  Off.  Appi  Hum  Engr  Rsch  Div.  Minitfry 
of  Defense,  New  Delhi 

1 Pars  Rsch  Ofc  Library,  AKA,  Israel  Defense  Forces 
1 Minister!*  van  Defensie,  DOOP/KL  Afd  Soctaal 
Psychol ogische  Zaken,  The  Hague,  Netherlands 
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